Statistical language learning

The $64,000 question in computational linguistics these days is: “What should I read to learn about statistical natural language processing?” I have been asked this question over and over, and each time I have given basically the same reply: there is no text that addresses this topic directly, and the best one can do is find a good probability-theory textbook and a good information-theory textbook, and supplement those texts with an assortment of conference papers and journal articles. Understanding the disappointment this answer provoked, I was delighted to hear that someone had finally written a bookdirectly addressing this topic. However, after reading Eugene Charniak’s Statistical Language Learning, I have very mixed feelings about the impact this bookmight have on the ever-growing field of statistical NLP. The book begins with a very brief description of the classic artificial intelligence approach toNLP (chapter 1), including morphology, syntax, semantics, and pragmatics. It presents a few definitions from probability theory and information theory (chapter 2), then proceeds to introduce hidden Markov models (chapters 3–4) and probabilistic context-free grammars (chapters 5–6). The book concludes with a few chapters discussing advanced topics in statistical language learning, such as grammar induction (chapter 7), syntactic disambiguation (chapter 8), word clustering (chapter 9), andword sense disambiguation (chapter 10). To its credit, the book serves as an interesting popular discussion of statistical modeling in NLP. It is well-written and entertaining, and very accessible to the reader with a limited mathematical background. It presents a good selection of statistical NLP topics to introduce the reader to the field. And the descriptions of the forward-backward algorithm for hidden Markov models and the inside-outside algorithm for probabilistic context-free grammars are intuitive and easy to follow. However, as a resource for someone interested in entering this area of research, this book falls far short of its author’s goals. These goals are clearly stated in the preface: